Counterfactual state explanations for reinforcement learning agents via generative deep learning

Counterfactual explanations, which deal with "why not?" scenarios, can provide insightful explanations to an AI agent's behavior. In this work, we focus on generating counterfactual for deep reinforcement learning (RL) agents operate in visual input environments like Atari. We introduce state a novel example-based approach based generative learning. Specifically, illustrates what minimal change is needed Atari game image such that the agent chooses different action. also evaluate effectiveness of states human participants who are not machine experts. Our first user study investigates if humans discern produced by actual or approach. second help non-expert identify flawed agent; compare against baseline nearest neighbor explanation uses images from game. results indicate have sufficient fidelity enable non-experts more effectively RL compared and having no at all.